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The  PRojection  Operator  based  Newton  method  for  Trajectory  Optimization 
(pronto)  is  an  iterative  algorithm  which,  in  its  simplest  form,  allows  one  to 
perform  local  Newton  (or  quasi-Newton)  optimization  of  the  cost  functional 

ftf 

h((x(-),u(-)))  :=  /  1(x(t),u(t),t)  dr  +  m(x(tf)) 

Jo 

over  the  set  of  trajectories  of  a  nonlinear  system  x  =  x  £  R",  u  £  Rm, 

subject  to  a  fixed  initial  condition  Xo ■  Here,  we  use  the  word  trajectory  in  an 
extended  sense  to  indicate  the  state-control  pair  77(f)  =  (x(t),u(t)):  t  >  0,  that 
satisfies  x(t)  =  f(x(t),u(t))  for  all  t  >  0.  As  usual,  “all  t”  means  “almost  all  t” 
in  the  sense  that  x(t)  =  x(0)  +  / (x(r) ,  u(t))  dr  where  /  ...  dr  is  the  Lebesgue 
integral.  The  cost  functional  h  above,  defined  in  terms  of  the  incremental  and 
terminal  costs  /(•,•)  and  m(-),  and  the  control  vector  field  /  are  taken  to  be 
sufficiently  smooth  (C2  in  (x,u)  and  continuous  in  t  is  usually  enough)  and 
regular  [3]. 

As  shown  in  [4],  the  set  T  of  trajectories  of  the  nonlinear  control  system 
x  =  f(x,u)  has  the  structure  of  a  (infinite  dimensional)  Banach  manifold,  a 
fact  that  allows  one  to  use  vector  space  operations  [9]  to  effectively  explore  it. 
To  work  on  the  trajectory  manifold  T,  one  projects  state-control  curves  in  the 
ambient  Banach  space  onto  T  by  using  a  local  linear  time-varying  trajectory 
tracking  controller.  To  this  end,  suppose  that  £  =  («(•), /x(-))  is  a  bounded 
state-control  curve  (an  approximate  trajectory)  and  let  77  =  (x(-),u(-))  be  the 
trajectory  of  x  =  f(x,u)  determined  by  the  nonlinear  feedback  system 

x{t)  =  f(x(t),u(t)), 

u(t)  =  p(t)  +  I\(t)(a(t)  -  x(t)), 

with  x(0)  =  xq.  Under  the  hypotheses  that  the  control  vector  field  /  is  Cr  and 
the  gain  K  is  bounded  [4] ,  this  feedback  system  defines  a  Cr  nonlinear  operator 

V  :£=  (a(-),  M'))  ^  V  =  (a(-)X‘))  • 

It  is  straightforward  to  see  that  £  is  a  fixed  point  of  P,  £  =  P(£),  F  and  only  if 
£  is  a  trajectory  of  the  control  system  x  =  f(x,u).  This  ensures  that  V2  =  V 
so  that  V  is  a  projection  operator.  With  this  projection  operator  at  hand,  one 
can  see  [3]  that  the  constrained  and  unconstrained  optimization  problems 

nhn/i(£)  and  minft(7:’(^)) 


are  essentially  equivalent  in  the  sense  that  a  solution  to  the  first  constrained 
problem  is  a  solution  to  the  second  unconstrained  problem,  while  a  solution  to 
the  second  problem  is,  projected  by  V,  a  solution  to  the  first  problem.  Using 
these  facts,  one  may  develop  Newton  and  quasi-Newton  descent  methods  for 
trajectory  optimization  in  an  effectively  unconstrained  manner  by  working  with 
the  cost  functional  g(£)  :=  h( P(£)). 

The  PRojection  Operator  based  Newton  method  for  Trajectory  Optimization 
(pronto)  is  given  by  [3] 

Algorithm  (Projection  operator  Newton  method) 
given  initial  trajectory  (o  €T 
for  i  =  0,  1,  2,  . . . 

redesign  feedback  K  if  desired/needed 

Ci  =  argminCeT5.r  Dh(£ ;)  •  <  +  \  D2g( &)  •  ((,  C)  ( search  direction ) 

7*  =  arg  min7e(o,i]  +  7C i)  (step  size) 

&+i  =  +  7»C»)  (update) 


end 

Note  that  the  functional  g(-)  and  the  projection  operator  V  depend  on  the  choice 
of  the  feedback  K.  Also,  Dg{^i)  and  D2g(£i)  are  the  first  and  second  Frecliet 
derivatives  of  the  Banach  space  functional  g.  When  (eT  and  (  £  TjT,  the 
first  derivative  Dg(£)  ■  (  simply  equals  Dh(£)  ■  (,  i.e. ,  it  does  not  depend  on  V. 

At  each  step,  the  minimization  of  a  second  order  approximation  of  the  ex¬ 
tended  cost  functional  g  provides  a  search  direction.  Then  an  optimal  step  size 
is  computed  through  a  (backtracking)  line  search  (a  pure  Newton  method  would 
use  a  fixed  step  size  of  y,  =  1).  Combining  the  search  direction  Q  with  step  size 
7i  a  new  update  trajectory  is  computed  and  the  algorithm  restarts  (unless  a  ter¬ 
mination  condition  is  met).  An  illustration  of  the  projection  operator  approach 
is  shown  in  Figure  1. 

The  computed  optimal  search  direction  Q  is  constrained  to  lie  on  the  tangent 
space  to  the  trajectory  manifold  at  the  current  iterate,  i.e.,  Q  £  Tj.T.  This  is 
not  restrictive  since,  as  established  in  [4,  Proposition  3.2],  V  can  be  used  to 
define  a  Injection  between  the  neighborhood  of  a  trajectory  £  £  T  and  the 
origin  of  its  tangent  space  T^T The  condition  Q  £  T^T  simply  means  that 
Q(t)  :=  (zi(t),Vi(t))  £  x  Rm,  t  >  0,  is  a  trajectory  of  the  linearization  of  the 
control  system  x  =  f(x,u)  about  the  current  trajectory  iterate  £j.  The  search 
direction  subproblem  is,  in  practice,  a  linear  quadratic  (LQ)  optimal  control 
problem,  where  the  functional  to  be  minimized,  Dh(£i)  ■  (  +  \D2g(£>i)  •  (£,£), 
is  the  quadratic  model  functional  given  by  the  first  two  terms  of  the  Taylor 
expansion  of  the  functional  g( £*  +  Q  with  respect  to  (  [3,  Section  3].  The 
LQ  problem  is  defined  using  first  and  second  order  derivatives  of  the  nonlinear 
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(a)  trajectory  manifold  (b)  search  direction 


Figure  1:  The  projection  operator  approach;  (a)  at  each  iteration,  the  lineariza¬ 
tion  of  the  control  system  about  the  trajectory  £,  defines  the  tangent  space  to 
the  trajectory  manifold  T  at  (b)  the  constrained  minimization  over  the  tan¬ 
gent  space  of  the  second  order  approximation  of  the  extended  cost  functional 
g  =  h  o  V  yields  the  search  direction  Q ;  (c)  the  optimal  step  size  is  computed 
through  a  line  search  along  £,;  (d)  the  search  direction  £*  and  step  size  7 *  are 
combined  to  obtain  a  new  update  trajectory  £,;+i . 


system  and  the  incremental  and  terminal  costs  about  the  current  (nonlinear 
system)  trajectory  iterate.  It  can  be  solved  by  computing  the  solution  to  a 
suitable  differential  Riccati  equation  (and  an  associated  adjoint  system).  In 
particular,  in  the  vector  space  case,  the  usual  chain  rule  applies  and  one  finds 
that  D2g( £)  ■  (£,  £)  is  a  well  defined  object  given  by 

£2ff(£MC,£)  =  D2h(0-(C,C)+Dh(0-D2V{0  ■  (£,£),  (1) 

for  £  £  T  and  £  £  TfT  [4],  Note  that  D2 V(tf)  is  the  second  Frechet  derivative 
of  the  Banach  space  operator  V . 

Our  work  has  been  focused  on  understanding  the  nature  of  highly  nonlinear 
dynamic  systems  and  especially  those  with  significant  maneuvering  objectives. 
We  believe  that  trajectory  optimization  provides  strong  tools  and  techniques  for 
discovering  and  understanding  important  dynamic  features  for  a  broad  range 
of  systems.  We  also  believe  that  it  is  only  by  doing  significant  numerical  ex¬ 
ploration  on  difficult  nonlinear  systems  that  we  begin  to  understand  how  such 
explorations  may  be  accomplished  through  the  use  of  appropriate  models  for 
the  systems  together  with  appropriate  cost  objectives  and  constraints.  To  this 
end  we  have  worked  with  systems  ranging  from  classical  nonlinear  pendulum 
systems  [1,  5]  to  air  [12,  15,  11],  land  [10,  23,  13,  2,  14,  24,  25],  and  marine  [7] 
vehicles,  and  even  earthquake  shaketables  [6].  Since  many  systems  of  interest 
do  not  evolve  in  a  flat  space,  we  have  devoted  significant  effort  to  understand¬ 
ing  and  extending  the  projection  operator  approach  to  work  with  manifold  and 
especially  with  Lie  groups  [18,  19,  20,  21,  16,  22,  17]. 
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Throughout  this  work,  we  have  developed  important  insights  into  the  na¬ 
ture  of  trajectory  functionals  and  their  minimization.  Second  order  properties 
of  the  trajectory  functionals  have  helped  us  to  discover  cases  where  one  would 
expect  a  functional  to  have  a  nice  minimizer  but  in  fact  one  does  not  exist.  Bar¬ 
rier  functionals  have  been  used  to  effectively  manage  input,  state,  and  mixed 
constraints.  Working  with  such  a  variety  of  systems  and  objectives,  we  have 
developed  a  sort-of  experts  toolkit  that  makes  such  investigations  (somewhat) 
more  tractable.  We  have  found  that,  in  order  to  do  effective  trajectory  explo¬ 
ration  and  optimization,  one  must  indeed  become  an  expert  on  the  system  under 
investigation! 


References 

[1]  Robert  Bailey  and  John  Hauser.  On  the  periodically  driven  inverted  pen¬ 
dulum.  In  Joint  48th  IEEE  Conference  on  Decision  and  Control  and  28th 
Chinese  Control  Conference,  pages  3142-3148,  Shanghai,  Dec.  2009. 

[2]  Florian  Bayer  and  John  Hauser.  Trajectory  optimization  for  vehicles  in 
a  constrained  environment.  In  51st  IEEE  Conference  on  Decision  and 
Control  (CDC),  Maui,  Dec.  2012. 

[3]  J.  Hauser.  A  projection  operator  approach  to  the  optimization  of  trajectory 
functionals.  In  Proceedinqs  of  the  15th  IFAC  World  Conqress ,  Barcelona, 
Spain,  2002. 

[4]  J.  Hauser  and  D.  G.  Meyer.  The  trajectory  manifold  of  a  nonlinear  con¬ 
trol  system.  In  Proceedings  of  the  37th  IEEE  Conference  of  Decision  and 
Control  (CDC),  volume  1,  pages  1034-1039,  1998. 

[5]  John  Hauser.  On  the  controllability  of  the  pendubot.  In  8th  IFAC  Sympo¬ 
sium  on  Nonlinear  Systems  (NOLCOS  2010),  Bologna,  Sept.  2010. 

[6]  John  Hauser  and  Mettupalayam  Sivaselvan.  On  the  computation  of  com¬ 
patible  trajectories  for  hydraulic  shaketables.  In  American  Control  Con¬ 
ference,  pages  5210-5215,  St.  Louis,  June  2009. 

[7]  Andreas  J.  Hausler,  Alessandro  Saccon,  A.  Pedro  Aguiar,  John  Hauser,  and 
Antonio  M.  Pascoal.  Cooperative  motion  planning  for  multiple  autonomous 
marine  vehicles.  In  9th  IFAC  Conference  on  Manoeuvring  and  Control  of 
Marine  Craft  (MCMC  2012),  Arenzano,  Italy,  Sept.  2012. 

[8]  Marcus  J.  Holzinger,  Daniel  J.  Scheeres,  and  John  Hauser.  Optimal  reacha¬ 
bility  sets  using  generalized  independent  parameters.  In  American  Control 
Conference  (ACC),  pages  905-912,  San  Francisco,  June  2011. 

[9]  David  G.  Luemberger.  Optimization  by  Vector  Space  Methods.  John  Wiley 
&  Sons,  1969. 


4 


[10]  Peter  MacMillin  and  John  Hauser.  Development  and  exploration  of  a  rigid 
motorcycle  model.  In  Joint  4 8th  IEEE  Conference  on  Decision  and  Control 
and  28th  Chinese  Control  Conference,  pages  4396-4401,  Shanghai,  Dec. 

2009. 

[11]  Giuseppe  Notarstefano  and  John  Hauser.  Computing  feasible  trajectories 
for  constrained  maneuvering  systems:  the  PVTOL  example.  Automatica, 
2011.  accepted. 

[12]  Guiseppe  Notarstefano  and  John  Hauser.  Modeling  and  dynamic  explo¬ 
ration  of  a  tilt-rotor  VTOL  aircraft.  In  8th  IFAC  Symposium  on  Nonlinear 
Systems  (NOLCOS  2010),  Bologna,  Sept.  2010. 

[13]  Alessandro  Rucco,  Giuseppe  Notarstefano,  and  John  Hauser.  Dynamics 
exploration  of  a  single-track  rigid  car  model  with  load  transfer.  In  49 th 
IEEE  Conference  on  Decision  and  Control  (CDC),  pages  4934-4939,  At¬ 
lanta,  Dec.  2010. 

[14]  Alessandro  Rucco,  Giuseppe  Notarstefano,  and  John  Hauser.  Computing 
minimum  lap-time  trajectories  for  a  single-track  car  with  load  transfer.  In 
51st.  IEEE  Conference  on  Decision  and  Control  (CDC),  Maui,  Dec.  2012. 

[15]  Enrico  Russo,  Giuseppe  Notarstefano,  and  John  Hauser.  Dynamics  explo¬ 
ration  and  aggressive  maneuvering  of  a  longitudinal  vectored  thrust  VTOL 
aircraft.  In  50th  IEEE  Conference  on  Decision  and  Control  and  European 
Control  Conference  (CDC-ECC),  pages  8106-8111,  Orlando,  Dec.  2011. 

[16]  Alessandro  Saccon,  A.  Pedro  Aguiar,  and  John  Hauser.  Lie  group  projec¬ 
tion  operator  approach:  Optimal  control  on  TSO(3).  In  50th  IEEE  Con¬ 
ference  on  Decision  and  Control  and  European  Control  Conference  ( CDC- 
ECC),  pages  6973-6978,  Orlando,  Dec.  2011. 

[17]  Alessandro  Saccon,  A.  Pedro  Aguiar,  Andreas  J.  Hausler,  John  Hauser,  and 
Antonio  M.  Pascoal.  Constrained  motion  planning  for  multiple  vehicles  on 
SE(3).  In  51st  IEEE  Conference  on  Decision  and  Control  (CDC),  Maui, 
Dec.  2012. 

[18]  Alessandro  Saccon,  John  Hauser,  and  A.  Pedro  Aguiar.  Exploration  of 
kinematic  optimal  control  on  the  Lie  group  SO(3).  In  8th  IFAC  Symposium 
on  Nonlinear  Systems  (NOLCOS  2010),  Bologna,  Sept.  2010. 

[19]  Alessandro  Saccon,  John  Hauser,  and  A.  Pedro  Aguiar.  On  the  closed-form 
solution  of  an  optimal  control  problem  on  the  Lie  group  SO (3).  In  Controlo 

2010,  Coimbra,  Portugal,  2010. 

[20]  Alessandro  Saccon,  John  Hauser,  and  A.  Pedro  Aguiar.  Optimal  control  on 
non-compact  Lie  groups:  A  projection  operator  approach.  In  49th  IEEE 
Conference  on  Decision  and  Control  (CDC),  pages  7111  7116,  Atlanta, 
Dec.  2010. 


5 


[21]  Alessandro  Saccon,  John  Hauser,  and  A.  Pedro  Aguiar.  Optimal  control  on 
Lie  groups:  Implementations  details  of  the  projection  operator  approach. 
In  18th  IFAC  World  Congress ,  Milano,  Aug.  2011. 

[22]  Alessandro  Saccon,  John  Hauser,  and  A.  Pedro  Aguiar.  Optimal  control 
on  Lie  groups:  The  projection  operator  approach.  IEEE  Transacations  on 
Automatic  Control,  2012.  accepted. 

[23]  Alessandro  Saccon,  John  Hauser,  and  Alessandro  Beghi.  A  dynamic  inver¬ 
sion  approach  to  motorcycle  trajectory  exploration.  In  Proceedings  of  the 
Bicycle  and  Motorcycle  Dynamics  Symposium  BMD2010,  2010. 

[24]  Alessandro  Saccon,  John  Hauser,  and  Alessandro  Beghi.  Trajectory  explo¬ 
ration  of  a  rigid  motorcycle  model.  IEEE  Transactions  on  Control  Systems 
Technology,  20(2):424-437,  2012. 

[25]  Alessandro  Saccon,  John  Hauser,  and  Alessandro  Beghi.  A  virtual  rider  for 
motorcycles:  Maneuver  regulation  of  a  multi-body  vehicle  model.  IEEE 
Transactions  on  Control  Systems  Technology ,  PP(99):1-15,  2012. 


6 


